Systems and methods for an immersive simulator

ABSTRACT

Systems, computer readable media, and method concern includes generating visual, auditory and other sensory depictions of a life-sized virtual crew member in an aircraft simulator. The virtual crew member simulates operational actions and behavioral and physiological responses of a crew member of an aircraft. The method includes collecting one or more responses of an operator using the aircraft simulator. The one or more responses comprise biofeedback data associated with the operator. The method includes generating, in response to the one or more responses, one or more simulated operational actions or simulated behavioral and physiological responses for the virtual crew member. The method includes collecting one or more additional responses of the operator. The one or more additional responses comprise additional biofeedback data of the operator that corresponds to an interaction with the one or more simulated operational actions or simulated behavioral and physiological responses for the virtual crew member.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application and claims priority toU.S. patent application Ser. No. 15/472,645 filed Mar. 29, 2017, thedisclosure of which is hereby incorporated herein by reference in itsentirety.

BACKGROUND

Currently, simulators are utilized to train students in the operation ofa vehicle and evaluate the operation of the vehicle. For example,aircraft simulators provide a simulated environment of an aircraft, suchas a cockpit, for training and performance evaluation. Currentsimulators, however, are limited to simulating the operation of thevehicle. Thus, there is a need for simulators that provide a completereal-world environment including interactions with passengers of thevehicle.

SUMMARY

Aspects of the present disclosure concern a method that includesgenerating visual, auditory and other sensory depictions of a life-sizedvirtual crew member in an aircraft simulator. The virtual crew membersimulates operational actions and behavioral and physiological responsesof a crew member of an aircraft. The method also includes collectingdata representing a state of the aircraft being simulated and one ormore responses of an operator using the aircraft simulator. The one ormore responses comprise biofeedback data associated with the operator.Additionally, the method includes generating, in response to at leastone of the data representing the state of the aircraft and the one ormore responses, one or more simulated operational actions or simulatedbehavioral and physiological responses for the virtual crew member.Further, the method includes collecting additional data representing anew state of the aircraft being simulated and one or more additionalresponses of the operator. The one or more additional responses compriseadditional biofeedback data of the operator that corresponds to aninteraction with the one or more simulated operational actions orsimulated behavioral and physiological responses for the virtual crewmember. The method also includes storing the one or more responses andthe one or more additional responses as a log of the operator's use ofthe aircraft simulator.

Additional aspects of the present disclosure concern a system thatincludes a simulator configured to simulate operation of a craft orvehicle. The simulator includes one or more controls and displays, oneor more biofeedback sensors, and a crew member simulator configured tosimulate a passenger or crew member of a craft or vehicle. The systemalso includes one or more processors coupled to the simulator. The oneor more processors are configured to execute the instructions, stored inone or more memory devices, to perform a method. The method includesgenerating, with the crew member simulator, visual, auditory and othersensory depictions of a life-sized virtual crew member in the simulator.The virtual crew member simulates operational actions and behavioral andphysiological responses of the passenger or crew member of the craft orvehicle. The method also includes collecting, from the one or morecontrols and displays and the one or more biofeedback sensors, datarepresenting a state of the craft or vehicle being simulated and one ormore responses of an operator using the simulator. The one or moreresponses comprise biofeedback data associated with the operator.Additionally, the method includes generating, in response to at leastone of the data representing the state of the craft or vehicle and theone or more responses, one or more simulated operational actions orsimulated behavioral and physiological responses for the virtual crewmember with the crew member simulator. The method also includescollecting, from the one or more controls and displays and the one ormore biofeedback sensors, additional data representing a new state ofthe craft or vehicle being simulated and one or more additionalresponses of the operator. The one or more additional responses compriseadditional biofeedback data of the operator that corresponds to aninteraction with the one or more simulated operational actions orsimulated behavioral and physiological responses for the virtual crewmember. Further, the method includes storing the one or more responsesand the one or more additional responses as a log of the operator's useof the simulator.

Additional aspects of the present disclosure concern a non-transitorycomputer readable medium storing instructions for causing one or moreprocessors to perform a method. The method includes generating visual,auditory and other sensory depictions of a life-sized virtual crewmember in an aircraft simulator. The virtual crew member simulatesoperational actions and behavioral and physiological responses of a crewmember of an aircraft. The method also includes collecting datarepresenting a state of the aircraft being simulated and one or moreresponses of an operator using the aircraft simulator. The one or moreresponses comprise biofeedback data associated with the operator.Additionally, the method includes generating, in response to at leastone of the data representing the state of the aircraft and the one ormore responses, one or more simulated operational actions or simulatedbehavioral and physiological responses for the virtual crew member.Further, the method includes collecting additional data representing anew state of the aircraft being simulated and one or more additionalresponses of the operator. The one or more additional responses compriseadditional biofeedback data of the operator that corresponds to aninteraction with the one or more simulated operational actions orsimulated behavioral and physiological responses for the virtual crewmember. The method also includes storing the one or more responses andthe one or more additional responses as a log of the operator's use ofthe aircraft simulator.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates an example of an immersive simulator, according tovarious aspects of the present disclosure.

FIG. 2 illustrates an example of an immersive aircraft simulator,according to various aspects of the present disclosure.

FIG. 3 illustrate an example of a process for operating an immersivesimulator, according to various aspects of the present disclosure.

FIG. 4 illustrate an example of a process for utilizing data generatedby an immersive simulator, according to various aspects of the presentdisclosure.

FIG. 5 illustrates an example of a hardware configuration for a computerdevice, according to various aspects of the present disclosure.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the principles of the presentteachings are described by referring mainly to examples of variousimplementations thereof. However, one of ordinary skill in the art wouldreadily recognize that the same principles are equally applicable to,and can be implemented in, all types of information and systems, andthat any such variations do not depart from the true spirit and scope ofthe present teachings. Moreover, in the following detailed description,references are made to the accompanying figures, which illustratespecific examples of various implementations. Logical and structuralchanges can be made to the examples of the various implementationswithout departing from the spirit and scope of the present teachings.The following detailed description is, therefore, not to be taken in alimiting sense and the scope of the present teachings is defined by theappended claims and their equivalents.

FIG. 1 illustrates an immersive simulator system 100 in which real-worldinteractions of crew members can be simulated and the interactions canbe recorded for training, according to aspects of the presentdisclosure. While FIG. 1 illustrates various components contained in theimmersive simulator system 100, FIG. 1 illustrates one example of asimulator system and additional components can be added and existingcomponents can be removed.

As illustrated in FIG. 1, the immersive simulator system 100 includes asimulator 102. The simulator 102 can be configured to provide asimulated and immersive environment of a vehicle for training andoperation by an operator 104. A vehicle can be a craft, an aircraft, amotor vehicle, a ship, a locomotive, a space craft. For example, thesimulator 102 can be an aircraft simulator, a motor vehicle simulator, aship simulator, a locomotive simulator, a space craft simulator, and thelike. The simulator 102 can include physical controls and displays,virtual controls and displays, and combinations thereof to simulate theoperation and environment of a craft or vehicle, as discussed below.

The simulator 102 includes controls and displays 106. The controls anddisplays 106 provide user interfaces for operating the simulator 102 andoutputting the responses and operations of the simulator 102 to theoperator 104. In some examples, the controls and displays 106 caninclude physical controls and displays, virtual controls and displays,and combination thereof to simulate operation of the craft or vehiclesimulated by the simulator 102. For example, if the simulator 102 is amotor vehicle simulator, the controls and displays 106 can includephysical controls and displays such as a steering wheel, accelerationpedal, brake pedal, gear controls, speed display, fuel level display,engine temperature display, audio controls and the like. Likewise, forexample, if the simulator 102 is a motor vehicle simulator, the controlsand displays 106 can include virtual controls and displays such as awindshield display that displays a virtual, real-time environment inwhich the motor vehicle is operating.

The simulator 102 includes a crew member simulator 108. The crew membersimulator 108 can be configured to simulate a crew member, passenger, orother live occupant (e.g., human) of a craft or vehicle simulated by thesimulator 102. The crew member simulator 108 can be configured tosimulate operational actions of the simulated crew member. Likewise, thecrew member simulator 108 can be configured to simulate physical,behavioral, physiological, and emotional actions and responses of thesimulated crew member. The crew member simulator 108 can be a physicaldevice or system, a virtual device or system, and combinations thereof.For example, the crew member simulator 108 can be a physical device orsystem that simulates the crew member, such as an animatronic robot.Likewise, for example, the crew member simulator 108 can be a virtualsystem such as a virtual reality display, a holographic generator orholographic display, and the like that projects an image of thesimulated crew member for viewing by the operator 104.

For example, if the simulator is a motor vehicle simulator and theoperator 104 is the driver of the simulated motor vehicle, the crewmember simulator 108 can simulate a crew member, or a passenger, of themotor vehicle. In this example, the crew member simulator 108 cansimulate the physical actions and responses of the simulated passengersuch as turning on the radio, changing environmental controls,conversing with the operator 104, and the like. Likewise, in thisexample, the crew member simulator 108 can simulate the emotionalactions or responses of the crew member, or passenger, such as asurprise or shock response to a simulated accident, anger over asimulated argument, and the like. As such, the operator 104 canexperience a complete, real-world simulation of operating the motorvehicle including real-world interactions with passengers of the motorvehicle. In an example, the crew member simulator 108 can simulate justthe emotional actions and responses of the crew member or,alternatively, the crew member simulator 108 can simulate just thephysical actions and responses of the simulated crew member.

The simulator 102 includes biofeedback sensors 110. The biofeedbacksensors 110 can capture signals representing conscious and subconsciousreactions, emotions, response, voice patterns, and the like of theoperator 104. The reactions, emotions, response, and the like can be inresponse to the operation of the simulator 102 and can be in response tothe interactions with the crew member simulated by the crew membersimulator 108. The biofeedback sensors 110 can be any type of devicethat is capable of sensing or detecting actions, reactions, inputs,emotions, and the like of the operator 104. For example, the biofeedbacksensors 110 can be configured to capture the biofeedback data includingspeech and sound data, scent data, brain wave data (e.g.,electroencephalogram (EEG) data), eye movement tracking data,micro-facial expressions data, body language, vital sign data (heartrate, blood pressure, respiration rate, etc.), muscle movement data,capillary dilation data, skin conductivity data, and the like.

The simulator 102 can also accommodate an instructor 112 via aninstructor station 114. The instructor station 114 can allow theinstructor 112 to initiate and control operation of the simulator 102.For example, the instructor station 114 can provide user interfaces toallow the instructor to select and configure different scenarios for thesimulator 102, select and configure simulated crew members, control andoperate the simulator 102, monitor and evaluate the performance of theoperator 104, provide feedback about the performance of the operator104, and the like. For example by providing a performance rating. Inanother example, the instructor station 114 can allow the instructor 112control and operate the crew member simulator 108 to communicate withthe operator 104 and directly control the operational actions and thephysical, behavioral, physiological, and emotional actions of thesimulated crew member.

The immersive simulator system 100 also includes a simulator controlsystem 116. The simulator control system 116 can be configured tocontrol the operation of the simulator 102 and the generation of thesimulated crew member by the crew member simulator 108. For example, thesimulator control system 116 can include hardware, software, andcombination thereof that communicate with the controls and displays 106to simulate the environment and operation of the craft or vehicle thatis the subject of the simulator 102. Likewise, for example, thesimulator control system 116 can include hardware, software, andcombination thereof that communicate with the biofeedback sensors 110 tocapture and analyze the biofeedback data of the operator 104. Also, forexample, the simulator control system 116 can include hardware,software, and combination thereof that communicate with the crew membersimulator 108 to generate the simulated crew member and operate thesimulated crew member, according to crew member models and in responseto the operational actions and biofeedback data from the operator 104.Additionally, for example, the simulator control system 116 can includehardware, software, and combination thereof that communicate with theinstructor station 114 to allow the instructor 112 to control andoperate the simulator 102.

As discussed above, the simulator control system 116 can be implementedas hardware, software, and combination thereof. For example, thesimulator control system 116 can be implemented as a simulator softwareprogram executing on one or more computer systems. The computer systemscan be any type of conventional computer system that is operating in theimmersive simulator system 100 or supporting the immersive simulatorsystem 100. For example, the computer systems can include various typesof servers, such as file servers, web servers, application servers,database servers, email servers and the like, that provide serviceswithin the immersive simulator system 100. Likewise, for example, thecomputer systems 102 can include laptop computers, desktop computers,tablet computers, mobile phones. In some examples, one or morecomponents of the simulator control system 116 can be implemented indedicated hardware.

Additionally, for example, the immersive simulator system 100 caninclude other hardware and computer systems that support the immersivesimulator system 100. For example, the immersive simulator system 100can include gateways, routers, wireless access points, firewalls, andthe like that support any type of communications network to allow thecomputing systems in the immersive simulator system 100 to communicate.In any of the examples, the computer systems in the immersive simulatorsystem 100 include hardware resources, such as processors, memory,network hardware, storage devices, and the like, and software resources,such as operating systems (OS), application programs, and the like.

In one example, the simulator control system 116 can include one or moresimulation software programs that controls the operation of thesimulator 102 and operation of the crew member simulator 108. Thesimulation software programs can be written in a variety of programminglanguages, such as JAVA, C++, Python code, Visual Basic, hypertextmarkup language (HTML), extensible markup language (XML), and the liketo accommodate a variety of operating systems, computing systemarchitectures, etc. In this example, the simulator software programs cancontrol the operation of the simulator 102. For instance, the simulatorsoftware programs can interface with the controls and displays 106 ofthe simulator 102 to simulate the craft or vehicle for the operator,such as run a simulation scenario for the simulator 102, display andupdate the operational status of the simulated craft or vehicle, displayand update the simulated environment in response to the operation by theoperator 104, and the like.

Likewise, for example, the simulator software programs can generate andcontrol the crew member simulator 108. The simulator software programscan generate, display, and control the simulated crew member on the crewmember simulator 108. The simulator software programs can generate,display, and control the simulated crew member on the crew membersimulator 108 autonomously based on one or more models of crew memberbehavior and actions, based on input from the instructor, andcombination thereof. For example, the simulator software programs cangenerate different types and configuration of simulated crew membersthat reflect different ages, genders, cultures, races, physicalappearance, and the like. Likewise, for example, the simulator softwareprograms can control and modify the simulated crew member on the crewmember simulator 108 so that the simulated crew member performs any oneof reacts to the operator 104, reacts to the simulation scenario,follows input from the instructor 112, and combination thereof.Additionally, for example, the simulator software programs can controland modify the simulated crew member on the crew member simulator 108 sothat the simulated crew member reacts to the biofeedback data of theoperator 104. For instance, the simulator software can control andmodify the simulated crew member to respond to the voice inflections ofthe operator 104, the speech phraseology of the operator 104, responseto stress detected in the operator 104, and the like.

The immersive simulator system 100 includes a repository 118. Therepository 118 can be configured to store data utilized by the simulator102 and the simulator control system 116. For example, the repository118 can store the operational parameters and data for the simulator thatallows simulation of a craft or vehicle. Likewise, the repository 118can store data generated by the simulator 102. For example, therepository 118 can store the biofeedback data captured by thebiofeedback sensors 110, the actions performed by the operator 104 onthe controls and displays 106, the actions performed by the simulatedcrew member, and input from the instructor 112 on the instructor station114.

The immersive simulator system 100 include one or more repositories 122.The computer system 102 can be coupled to one or more repositories 122via one or more networks 120. For example, the repositories 122 can beoperated and maintained by the immersive simulator system 100 and/orother entities. The network 120 can be any type of network whetherpublic or private. The repositories 122 can be configured to store datautilized by the simulator 102 and the simulator control system 116. Forexample, the repositories 122 can store the operational parameters anddata for the simulator that allows simulation of a craft or vehicle.Likewise, the repositories 122 can store data generated by the simulator102. For example, the repositories 122 can store the biofeedback datacaptured by the biofeedback sensors 110, the actions performed by theoperator 104 on the controls and displays 106, the actions performed bythe simulated crew member, and input from the instructor 112 on theinstructor station 114.

The immersive simulator system 100 includes a training system 124. Thetraining system 124 can be coupled to the repository 118 (and/or therepositories 122) to retrieve the data captured for the operator 104.The training system 124 can provide the data to the operator 104. Forexample, the training system 124 can provide a training summary to theoperator 104 that includes a performance evaluation or rating of theoperation of the simulator 102 and the interactions with the simulatedcrew member.

Using the components discussed above, the immersive simulator system 100provide a standalone and/or instructor-influenced learning system thatallows interjection of leadership, simulation and instruction ofcommunication, crew resource management and interactive skill learning.The crew member simulator 108 provides an interactive simulation thatinvokes emotion and confrontation (e.g., disrespecting authority, lackof cooperation, beratement, embarrassment, etc.), for the operator 104in simulator 102 and focuses on communication and other crew resourcemanagement competencies through verbal speech and body language toinclude facial expressions. As such, the immersive simulator system 100can teach or allow practice of coping mechanisms and allow developmentof a mastery in personal skills in addition to operation training andevaluation. Additionally, the immersive simulator system 100 providesafter action review to emphasize learning points.

As discussed above, the immersive simulator system 100 can provide asimulated and immersive environment of a craft or vehicle. FIG. 2illustrates one example of an immersive simulator system 100, animmersive aircraft simulator system 200, in which real-worldinteractions of aircraft crew members can be simulated and theinteractions can be recorded for training, according to aspects of thepresent disclosure. While FIG. 2 illustrates various componentscontained in the immersive aircraft simulator system 200, FIG. 2illustrates one example of an aircraft simulator and additionalcomponents can be added and existing components can be removed.

The immersive aircraft simulator system 200 includes a flight deck 202.The flight deck 202 can be configured to simulate the environment of anaircraft flight deck. The flight deck 202 can include the systems,hardware, control, displays, and the like found on an aircraft flightdeck. In examples, the flight deck 202 can be a flight training deviceand/or full flight simulator. Flight deck 202 can be implemented usingany commercially available flight deck simulator along withmodifications according to aspects described herein. In some examples,the flight deck 202 can be implemented using a flight deck simulatoravailable from CAE, Inc. and/or Thales.

The flight deck 202 includes a pilot station 204. The pilot station 204can include the systems, hardware, control, displays, and the like thatsimulates a pilot's control and operation of a simulated aircraft. Forexample, the pilot station 204 can include a seat, control stick oryolk, controls, flaps and elevator controls, electronic flight bag, andthe like. Additionally, for example, the pilot station 204 can includedisplays such as a primary flight display, multifunction display,navigation display, head up display, and engine indicating and crewalerting system (EICAS) display. The primary flight display can provideinformation such as, for example, airspeed, altitude, heading, attitude,vertical speed, and glideslope. The multifunction display can providestatus information about aircraft systems such as, for example,communications, auxiliary power unit (APU), fuel, and electricalsystems. The navigation display can present heading/track, flight planroute, waypoint, airport, navigational aid and weather information. Thehead up display (HUD) can provide projected airplane flight data on theHUD combiner mounted just in front of the windshield. The EICAS displaycan present information about aircraft systems, including propulsion,warnings, flight controls, fuel, and air systems.

The flight deck 202 includes one or more devices and systems to capturebiofeedback data. The biofeedback sensors can be any type of system thatincludes hardware, software, or combinations thereof and is capable ofsensing or detecting actions, reactions, inputs, emotions, and the likeof the operator 104. For example, the biofeedback sensors can beconfigured to capture the biofeedback data including speech and sounddata, scent data, brain wave data (e.g., electroencephalogram (EEG)data), eye movement tracking data, micro-facial expressions data, bodylanguage, vital sign data (heart rate, blood pressure, respiration rate,etc.), muscle movement data, capillary dilation data, skin conductivitydata, and the like.

For example, the biofeedback devices and system can include a facial andgesture capture system 206, a voice recognition system 208, and aphysiological measurement system 210. The facial and gesture capturesystem 206 can be any type of system that includes hardware, software,or combinations thereof to detect, capture, and analyze facial gesturesand body movements of the operator so that emotional and cognitivestates of the operator can be determined. The voice recognition system208 can be any type of system that includes hardware, software, orcombinations thereof to detect, capture, and analyze the speech andsounds emitted by an operator or student. For instance, the voicerecognition system 208 can analyze speech and sounds of the operator orstudent to detect the syntax and spoken words of the operator or studentin order to identify speech directed to the simulation scenario anddifferentiate between actionable and non-actionable commands. Likewise,for instance, the voice recognition system 208 can analyze speech andsounds of the operator or student to detect the emotional aspects (e.g.,inflection, tone, etc.) and interpolate the operator or studentemotional state. The physiological measurement system 210 can be anytype of system that includes hardware, software, or combinations thereofto detect, capture, and analyze physiological states of the operator orstudent (e.g., brainwaves, vital signs, capillary dilation, skinconductivity, etc.)

The flight deck 202 includes a co-pilot simulator 212. The co-pilotsimulator 212 can be configured to simulate a co-pilot of an aircraftsimulated by the simulator 102. The co-pilot simulator 212 can beconfigured to simulate the operational actions of the simulatedco-pilot. Likewise, the co-pilot simulator 212 can be configured tosimulate physical, behavioral, physiological, and emotional actions andresponses of the simulated so-pilot. The co-pilot simulator 212 can be aphysical device or system, a virtual device or system, and combinationsthereof. For example, the co-pilot simulator 212 can be a physicaldevice or system that simulates the co-pilot, such as an animatronicrobot. Likewise, the co-pilot simulator 212 can be a virtual system suchas a virtual reality display, an augmented reality display, aholographic generator, and the like that projects an image of thesimulated crew member for the operator 104. While co-pilot simulator 212is described as simulating a co-pilot, the co-pilot simulator 212 can beconfigured to simulate any crew member, passenger, or other liveoccupant (e.g., human) of an aircraft.

In one example, the co-pilot simulator 212 can include a scent generator214, a display system 216, and an audio system 218. The scent generator214 can be any type of system that includes hardware, software, orcombinations thereof to generate scents and smells types found in aflight deck in different simulated scenarios and/or emitted from a crewmember in different simulated scenarios. The display system 216 can beany type of system that includes hardware, software, or combinationsthereof to display an image or avatar of the simulated crew member. Forexample, the display system 216 can be a virtual reality display (e.g.,Hololens™, Occulus™, HTC Vive™, etc.), a conventional display or monitor(e.g., cathode ray tube display, liquid crystal display, plasma display,light emitting diode display, etc.), a holographic emitter, and thelike. The audio system 218 can be can be any type of system thatincludes hardware, software, or combinations thereof to generate soundsfound in a flight deck in different simulated scenarios and/or emittedfrom a crew member in different simulated scenarios. For example, theaudio system 218 can be a speaker system, a headset (e.g., pilotscommunication headset), and the like.

The flight deck 202 includes an instructor station 226. The instructorstation 226 can allow the instructor to initiate and control operationof the flight deck 202. For example, the instructor station 226 canprovide user interfaces to allow the instructor to select and configuredifferent scenarios for the flight deck 202, select and configure thesimulated co-pilot, control and operate the flight deck 202, monitor andevaluate the performance of the operator or student, provide feedbackabout the performance of the operator or student, and the like. Inanother example, the instructor station 226 can allow the instructor tocontrol and operate the co-pilot simulator 212 to communicate with theoperator and directly control a portion, or all, of the operationalactions and the physical, behavioral, physiological, and emotionalactions of the simulated co-pilot.

The co-pilot simulator 212 and other systems, devices, and controls ofthe flight deck 202 are coupled to a co-pilot control system 220. Theco-pilot control system 220 can include hardware, software, andcombination thereof that communicate with the co-pilot simulator 212 togenerate the simulated crew member and operate the simulated crew memberaccording to crew member scenarios and in response to the operationalactions and biofeedback data from the operator or student. The co-pilotcontrol system 220 can generate, display and control the simulatedco-pilot with the co-pilot simulator 212 autonomously based on one ormore models of crew member behavior and action, based on input from theinstructor, and combination thereof. For example, the co-pilot controlsystem 220 can generate different types and configuration of simulatedco-pilot that reflect different ages, genders, cultures, races, physicalattributes, and the like. Likewise, for example, the co-pilot controlsystem 220 can control and modify the simulated co-pilot with theco-pilot simulator 212 so that the simulated crew member reacts to theoperator or student, reacts to the simulation scenario, follows inputfrom the instructor, and combination thereof. Additionally, for example,the co-pilot control system 220 can control and modify the simulatedco-pilot with the co-pilot simulator 212 so that the simulated co-pilotreacts to the biofeedback data of the operator or student. For instance,the co-pilot control system 220 can control and modify the simulatedco-pilot to respond to the voice inflections of the operator or student,the speech phraseology of the operator or student, response to stressdetected in the operator or student, and the like.

The co-pilot control system 220 can include avatar physical model 222and an avatar intelligent behavior model 224. The avatar physical model222 defines logic and data to support the physical rendering of thesimulated co-pilot generated and displayed by the co-pilot simulator212. The co-pilot control system 220 can maintain, store, and utilizedifferent versions of the avatar physical model 222 that reflectdifferent ages, genders, cultures, races, physical attributes, and thelike. The avatar physical model 222 also defines motor movements,speech, head movements, facial responses, and the like that provide aperception of the simulated co-pilot interacting with the flight deck202. The avatar physical model 222 also defines motor movements, speech,head movements, facial responses, and the like that provide realisticspeech of the simulated co-pilot interacting with the operator orstudent. The avatar physical model 222 also defines motor movements,speech, head movements, facial responses, and the like that provide aperception of emotion in the simulated co-pilot.

The avatar intelligent behavior model 224 defines logic and data tosupport behavior and responses in order to simulate a cognitive state inthe simulated co-pilot and generate realistic actions (e.g.,conversation, emotions, physical task, etc.) to simulate the co-pilot inthe scenarios. For instance, the avatar intelligent behavior model 224can define behavior and responses to simulate expert pilot or co-pilotcognition and comprehension of flight deck 202 procedures andcommunications. Likewise, for instance, the avatar intelligent behaviormodel 224 can define behavior and responses that simulate actions inaccordance with simulation scenarios and performance measurecheckpoints. Also, for instance, the avatar intelligent behavior model224 can define branching logic to invoke situations in the flight deck202 (e.g., flight deck faults, equipment failures, crew communicationissues, etc.) to allow measurement of the operator or studentcompetencies. Further, for instance, the avatar intelligent behaviormodel 224 can define logic to recognize the cognitive and emotionalstate of the operator or student and respond to the cognitive andemotional state of the operator or student.

The avatar intelligent behavior model 224 (and/or the avatar physicalmodel 222) also defines data output to support the operation of theimmersive aircraft simulator system 200. For example, the avatarintelligent behavior model 224 (and/or the avatar physical model 222)outputs data to the co-pilot simulator 212 in order to respond to and togenerate verbal communication, for instance, syntax and spoken words andvoice inflection to indicate realistic behavior and emotion. Also, forexample, the avatar intelligent behavior model 224 (and/or the avatarphysical model 222) outputs data to the co-pilot simulator 212 in orderto respond to and to generate non-verbal communication (e.g., facialexpressions, gestures, etc.). Additionally for example, the avatarintelligent behavior model 224 (and/or the avatar physical model 222)outputs data to the co-pilot simulator 212 in order to generateolfactory cues (e.g., variance in hygiene, bodily function, etc.) andgenerate physical movement. The avatar intelligent behavior model 224(and/or the avatar physical model 222) also receives input in order tosimulate real-world interactions. For example, the avatar intelligentbehavior model 224 (and/or the avatar physical model 222) can any one ofreceive data from the facial and gesture capture system 206, receivedata from the voice recognition system 208, receive data from thephysiological measure system 210, receive data from systems controllingthe simulation scenario, receive data from an instructor or other user,and any combination thereof, and the like.

The immersive aircraft simulator system 200 also includes a crew stationsimulation system 228. The crew station simulation system 228 can becoupled to the pilot station 204 and the instructor station 226. Thecrew station simulation system 228 can be configured to manage andcontrol the input and output of data to the pilot station 204 tosimulate conditions in the flight deck 202.

The immersive aircraft simulator system 200 also includes a scenariomanager 230. The scenario manager 230 can be coupled to the co-pilotcontrol system 220, the crew station simulation system 228, a standaloneoperator console 232, and a performance measurement system 234. Thescenario manager 230 includes logic, commands, instructions, and thelike to provide and mange scenario data, vehicle simulation events, andstate data provided to the components of the immersive aircraftsimulator system 200. For example, the scenario manager 230 can providean interface with a data daemon of the crew station simulation system228 to read and write simulation data and load scenario profiles. Also,for example, the scenario manager 230 can provide an interface to theco-pilot control system 220 to provide simulation and scenario data andfeedback simulation trigger event to the crew station simulation system228. Additionally, for example, the scenario manager 230 can provide aninterface to the performance measurement system 234 to providesimulation data, operator or student actions, instructor input, and thelike. Also, for example, the scenario manager 230 can provide aninterface to the standalone operator console 232 to display data fromthe simulation and trigger override of intelligent agent actions.

The standalone operator console 232 can provide an interface for aninstructor or other user to initiate, configure, and operate thecomponents of the immersive aircraft simulator system 200. Thestandalone operator console 232 can include hardware, software, andcombination thereof to provide the interface to the instructor or otheruser.

The performance measurement system 234 collects the data generated bythe immersive aircraft simulator system 200 and determines a performancemeasure for the operator or student. The performance measurement system234 can provide a user interface to create objective measure of studentperformance. For example, the performance measurement system 234 canprovide a user interface to specify algorithms that use simulator datathat evaluate the operator or student performance against a predefinedstandard. For example, a standard can be a predetermined level ofperformance in the combination of operating the aircraft simulator andbiofeedback, and interactions with the virtual crew member. Also, forexample, the performance measurement system 234 can provide a userdefined trigger event to define when metrics for the operator or studentshould be evaluated. The performance measurement system 234 includeslogic to use simulation data to perform evaluation of predefinedmeasures of performance. For example, the performance measurement system234 can include logic to collect and annotate data from the immersiveaircraft simulation system 200 to include ownership of performancecharacteristics (e.g., speed, altitude, etc.). For example, theperformance measurement system 234 can include logic to collect,evaluate, and annotate the operator or student interaction with theflight deck 202 (e.g., button presses, throttle control) and interactionwith the simulated co-pilot. The performance measurement system 234includes logic to send performance measures to other systems such as thestandalone operator console 232 and a feedback and logging system 236.

The immersive aircraft simulator system 200 also includes the feedbackand logging system 236. The feedback and logging system 236 can becoupled to the co-pilot control system 220, the performance measurementsystem 234, and a training system 238. The feedback and logging system236 provides performance feedback to the operator or student, forexample, via the training system 238. The feedback and logging system236 can automatically provide verbal or visual indications ofperformance and progress, automatically provide verbal and visualrecommendations of action to improve performance, and the like. Also,for example, the feedback and logging system 236 can automaticallyhighlight an area in the simulation (e.g., altimeter) to direct theoperator or student to focus on an instrument or other part of theflight deck 202 to correct performance. Also, for example, the feedbackand logging system 236 can automatically adjust the simulator (e.g.,adjust speed or altitude) based on the assessment of the operator orstudent performance. Also, for example, the feedback and logging system236 can generate and provide a training summary such as a graphical,tabular, numerical indication of the operator or student performanceincluding the performance measures results. The feedback and loggingsystem 236 also stores any data from the immersive aircraft simulator200, the training summary, and the performance measures.

FIG. 3 illustrates an example of a process for providing and operatingan immersive simulator, according to aspects of the present disclosure.While FIG. 3 illustrates various stages that can be performed, stagescan be removed and additional stages can be added. Likewise, the orderof the illustrated stages can be performed in any order.

After the process begins, in 302, simulation parameters are determined.The simulation parameters can include any data to initiate and operatethe immersive simulator system 100 or the immersive aircraft simulatorsystem 200. For example, the simulation parameters can include asimulation scenario, configuration parameters for the simulation, modelsfor the crew member simulator, configuration parameters for the crewmember simulator, and the like. In examples, one or more of thesimulation parameters can be input by the instructor 112 (or otheruser). In example, one or more of the simulation parameters can bepredefined in the immersive simulator system 100 or the immersiveaircraft simulator system 200.

In 304, the simulation including the simulated crew member or virtualcrew member is initiated. Once the simulation parameters are determined,the simulator control system 116 can initiate the simulator 102. Forexample, the simulator control system 116 can initiate the controls anddisplays 106 with initial status information based on the simulationscenario and begin receiving input from the operator 104. Additionally,for example, the simulator control system 116 can initialize thebiofeedback sensors 110 and begin receiving biofeedback data from theoperator 104. Additionally, for example, the simulator control system116 can initialize the crew member simulator 106 based on the crewmember models and begin generate the simulated crew member forinteraction with the operator 104. Additionally, for example, thesimulator control system 116 can initialize the instructor station 114and begin receiving input from the instructor 112. Additionally, forexample, the simulator control system 116 can begin storing data fromthe simulator 102 in the repository 118 and/or the repositories 122.

In 306, the immersive simulator system generates simulated operationalactions or simulated behavioral and physiological responses for thevirtual crew member. For example, based on the simulation scenario, thesimulation control system 116 generates and modifies the appearance andoperational action to conform to the operation of the simulator 102. Forinstance, in a simulation for the immersive aircraft simulator system200, the co-pilot control system 220 generates and modifies theappearance and operational actions, for example, interacting inpreflight checks with the operator or student, moving or operatingcontrols, conversing with the operator or student, and the like.Additionally, for instance, in a simulation for the immersive aircraftsimulator system 200, the co-pilot control system 220 generates andmodifies simulated behavioral and physiological responses for thevirtual crew member, for example, facial expressions and increasedvisible sweat on the crew member's forehead.

In 308, the immersive simulator system collects responses of theoperator including biofeedback data. In 310, the immersive simulatorsystem stores responses of the operator, simulated operation actions orsimulated behavioral and physiological responses for the virtual crewmember, and data for the simulation. For example, the simulator controlsystem 116 can capture input from the operator 104, for instance,interactions with the controls and displays 106. Additionally, forexample, the simulator control system 116 receive, from the biofeedbacksensors 110, biofeedback data from the operator 104. Additionally, forexample, the simulator control system 116 can record the simulatedoperational action or simulated behavioral and physiological responsesfor the virtual crew member. Additionally, for example, the simulatorcontrol system 116 can receive input from the instructor 112.Additionally, for example, the simulator control system 116 can storedata from the simulator 102 in the repository 118 and/or therepositories 122.

In 312, the immersive simulator system determines whether to continuethe simulation. For example, the simulation can continue for apredetermined amount of time, until a simulation scenario is completed,until stopped by an event (e.g., crash of the craft or vehicle), untilstopped by an instructor or other user, and the like. If the simulationis complete, the immersive simulator system can generate a trainingsummary of the simulation. The training summary can include data fromthe simulation such as the action of the operator 104, interactions withthe simulated crew member, operational and simulation parameters, avideo and audio recording of the simulation, and the like. The trainingsummary can also include one or more performance metrics. Theperformance metrics can be performance metrics associated with theoperation of the simulator 102, associated with interaction with thesimulated crew member, and the like. The performance metrics can beautomatically generated by the immersive simulator system 100 or theimmersive aircraft simulator system 200, can be based at least partiallyon input from an instructor or other user, and the like. After, themethod 300 can end.

If the simulation continues, in 314, the immersive simulator system canoptionally receive input from an instructor. For example, the instructor112 can provide input regarding the performance of the operator 104,events to occur during the simulation, behavior and action of thesimulated crew member, and the like.

In 316, the immersive simulator system can optionally modify the virtualcrew member. For example, the appearance, emotion state, actions and thelike can be modified. The virtual crew member can be modified based onthe biofeedback data from the operator 104, input form the instructor112, events in the simulation scenario, and the like. After, the method300 return to 306 and repeats.

FIG. 4 illustrates an example of a process for utilizing data generatedby an immersive simulator, according to aspects of the presentdisclosure. While FIG. 4 illustrates various stages that can beperformed, stages can be removed and additional stages can be added.Likewise, the order of the illustrated stages can be performed in anyorder.

After the process begins, in 402, data is received that includesresponses of the operator, simulated operational actions or simulatedbehavioral and physiological responses for the virtual crew member, anddata for the simulation. In one example, the simulator control system116 can receive the data. In another example, the performancemeasurement system and/or the feedback and logging system 236 canreceive the data.

In 404, optionally, input can be received from an instructor. The inputcan include performance evaluation of the operator or student duringoperation of the simulator 102 or flight deck 202, interaction with thesimulated crew member or simulated co-pilot, and the like. For example,the instructor 112 can provide input regarding the performance of theoperator 104, events to occur during the simulation, behavior and actionof the simulated crew member, and the like.

In 406, the responses of the operator, simulated operational actions orsimulated behavioral and physiological responses for the virtual crewmember, and data for the simulation are compared to an operationalstandard associated with the simulation. In 408, a performance metric isdetermined based at least partially on the comparison, the input fromthe instructor, or both.

For example, the simulator control system 116 (or the performancemeasurement system 234) collects the data generated by the immersivesimulator system 100 (the immersive aircraft simulator system 200) anddetermines performance measures for the operator or student. Thesimulator control system 116 (or the performance measurement system 234)can utilize objective measure of student performance. For example, thesimulator control system 116 (or the performance measurement system 234)can utilize algorithms that use simulator data that evaluate theoperator or student performance against a predefined standard,instructions input, and combination thereof. Also, for example, thesimulator control system 116 (or the performance measurement system 234)can utilize trigger events to define when metrics for the operator orstudent should be evaluated. For example, the simulator control system116 (or the performance measurement system 234) can collect and annotatedata from the immersive aircraft simulation system 200 to includeownership of performance characteristics (e.g., speed, altitude, etc.),the operator or student interaction with the flight deck 202 (e.g.,button presses, throttle control) and interaction with the simulatedco-pilot.

For example, the feedback and logging system 236 can automaticallyprovide verbal or visual indications of performance and progress,automatically provide verbal and visual recommendations of action toimprove performance, and the like. Also, for example, the feedback andlogging system 236 can automatically highlight an area in the simulation(e.g., altimeter) to direct the operator or student to focuses on aninstrument or other part of the flight deck 202 to correct performance.Also, for example, the feedback and logging system 236 can automaticallyadjust the simulator (e.g., adjust speed or altitude) based on theassessment of the operator or student performance.

In 410, a training summary is generated that includes the performancemetric. The training summary can include data from the simulation suchas the action of the operator 104, interactions with the simulated crewmember, operational and simulation parameters, a video and audiorecording of the simulation, and the like. The training summary can alsoinclude one or more performance metrics. In examples, the video andaudio recording of the simulation can include verbal or visualindications of performance and progress, highlighted areas in thesimulation to direct the operator or student to focuses on an instrumentor other part of the flight deck to correct performance, and the like.

In 412, the virtual crew member is updated for future simulations basedat least partially on the training summary. For example, based on theoperator's interaction with simulated crew member, the models of thesimulated crew member can be updated to improve the real-worldappearance and interactions of the simulated crew member.

In 414, the training summary is provided to the operator. In oneexample, the training summary can be provided via the training system124. In another example, the training summary can be provided via thetraining system 238.

The foregoing description is illustrative, and variations inconfiguration and implementation can occur to persons skilled in theart. For instance, the various illustrative logics, logical blocks,modules, and circuits described in connection with the embodimentsdisclosed herein can be implemented or performed with a general purposeprocessor, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A general-purpose processor canbe a microprocessor, but, in the alternative, the processor can be anyconventional processor, controller, microcontroller, or state machine. Aprocessor can also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration.

In one or more exemplary embodiments, the functions described can beimplemented in hardware, software, firmware, or any combination thereof.For a software implementation, the techniques described herein can beimplemented with modules (e.g., procedures, functions, subprograms,programs, routines, subroutines, modules, software packages, classes,and so on) that perform the functions described herein. A module can becoupled to another module or a hardware circuit by passing and/orreceiving information, data, arguments, parameters, or memory contents.Information, arguments, parameters, data, or the like can be passed,forwarded, or transmitted using any suitable means including memorysharing, message passing, token passing, network transmission, and thelike. The software codes can be stored in memory units and executed byprocessors. The memory unit can be implemented within the processor orexternal to the processor, in which case it can be communicativelycoupled to the processor via various means as is known in the art.

For example, FIG. 5 illustrates an example of a hardware configurationfor the computer system that can be used in any of the components of theimmersive simulator system 100 or immersive simulator system 200. WhileFIG. 5 illustrates various components contained in the computer device500, FIG. 5 illustrates one example of a computer device and additionalcomponents can be added and existing components can be removed.

The computer device 500 can be any type of computer device. Asillustrated in FIG. 5, the computer device 500 can include one or moreprocessors 502 of varying core configurations and clock frequencies. Thecomputer device 500 can also include one or more memory devices 504 thatserve as a main memory during the operation of the computer device 500.For example, during operation, a copy of the software that supports thesimulation can be stored in the one or more memory devices 504. Thecomputer device 500 can also include one or more peripheral interfaces506, such as keyboards, mice, touchpads, computer screens, touchscreens,etc., for enabling human interaction with and manipulation of thecomputer device 500.

The computer device 500 can also include one or more network interfaces508 for communicating via one or more networks, for example the network120, such as Ethernet adapters, wireless transceivers, or serial networkcomponents, for communicating over wired or wireless media usingprotocols. The computer device 500 can also include one or more storagedevices 510 of varying physical dimensions and storage capacities, suchas flash drives, hard drives, random access memory, etc., for storingdata, such as images, files, and program instructions for execution bythe one or more processors 502.

Additionally, the computer device 500 can include one or more softwareprograms 512 that enable the functionality of the simulation environmentdescribed above. The one or more software programs 512 can includeinstructions that cause the one or more processors 502 to perform theprocesses described herein. Copies of the one or more software programs512 can be stored in the one or more memory devices 504 and/or on in theone or more storage devices 510. Likewise, the data utilized by one ormore software programs 512 can be stored in the one or more memorydevices 504 and/or on in the one or more storage devices 510.

The computer device 500 can include a variety of data stores and othermemory and storage media as discussed above. These can reside in avariety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network. In some implementations, informationcan reside in a storage-area network (SAN) familiar to those skilled inthe art. Similarly, any necessary files for performing the functionsattributed to the computers, servers, or other network devices may bestored locally and/or remotely, as appropriate.

In implementations, the components of the computer device 500 asdescribed above need not be enclosed within a single enclosure or evenlocated in close proximity to one another. Those skilled in the art willappreciate that the above-described componentry are examples only, asthe computer device 500 can include any type of hardware componentry,including any necessary accompanying firmware or software, forperforming the disclosed implementations. The computer device 500 canalso be implemented in part or in whole by electronic circuit componentsor processors, such as application-specific integrated circuits (ASICs)or field-programmable gate arrays (FPGAs).

If implemented in software, the functions can be stored on ortransmitted over a computer-readable medium as one or more instructionsor code. Computer-readable media includes both tangible, non-transitorycomputer storage media and communication media including any medium thatfacilitates transfer of a computer program from one place to another. Astorage media can be any available tangible, non-transitory media thatcan be accessed by a computer. By way of example, and not limitation,such tangible, non-transitory computer-readable media can comprise arandom access memory (RAM), a read only memory (ROM), a flash memory, anelectrically erasable programmable read only memory (EEPROM), a compactdisc read only memory (CD-ROM) or other optical disk storage, magneticdisk storage or other magnetic storage devices, or any other medium thatcan be used to carry or store desired program code in the form ofinstructions or data structures and that can be accessed by a computer.Disk and disc, as used herein, includes CD, laser disc, optical disc,digital versatile disc (DVD), floppy disk and Blu-ray disc where disksusually reproduce data magnetically, while discs reproduce dataoptically with lasers. Also, any connection is properly termed acomputer-readable medium. For example, if the software is transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. Combinations of the above should also be includedwithin the scope of computer-readable media.

While the teachings have been described with reference to examples ofthe implementations thereof, those skilled in the art will be able tomake various modifications to the described implementations withoutdeparting from the true spirit and scope. The terms and descriptionsused herein are set forth by way of illustration only and are not meantas limitations. In particular, although the processes have beendescribed by examples, the stages of the processes can be performed in adifferent order than illustrated or simultaneously. Furthermore, to theextent that the terms “including”, “includes”, “having”, “has”, “with”,or variants thereof are used in the detailed description, such terms areintended to be inclusive in a manner similar to the term “comprising.”As used herein, the terms “one or more of” and “at least one of” withrespect to a listing of items such as, for example, A and B, means Aalone, B alone, or A and B. Further, unless specified otherwise, theterm “set” should be interpreted as “one or more.” Also, the term“couple” or “couples” is intended to mean either an indirect or directconnection. Thus, if a first device couples to a second device, thatconnection can be through a direct connection, or through an indirectconnection via other devices, components, and connections.

What is claimed is:
 1. A method, comprising: generating sensorydepictions of a life-sized virtual crew member in an aircraft simulator,wherein the virtual crew member simulates operational actions andresponses of a crew member of an aircraft; collecting data representinga state of the aircraft being simulated and one or more responses of anoperator during operation of the aircraft simulator, wherein the one ormore responses comprise biofeedback data associated with the operator;generating, in response to at least one of the data representing thestate of the aircraft and the one or more responses, one or moresimulated operational actions or simulated responses for the virtualcrew member; and storing the one or more responses as a log of theoperator's operation of the aircraft simulator.
 2. The method of claim1, the method further comprising: updating the one or more simulatedoperational actions or simulated responses of the virtual crew memberbased at least partially on the log of the operator's operation of theaircraft simulator.
 3. The method of claim 1, the method furthercomprising: receiving instructor input, wherein the one or moresimulated operational actions or simulated responses for the virtualcrew member are based at least partially on the instructor input.
 4. Themethod of claim 1, the method further comprising: generating a trainingsummary based at least partially on the log of the operator's operationof the aircraft simulator.
 5. The method of claim 4, the method furthercomprising: comparing the one or more responses to an operationalstandard of the aircraft simulator; determining, based at leastpartially on the comparing, a performance rating of the operator'soperation of the aircraft simulator; and providing the performancerating as a portion of the training summary.
 6. The method of claim 5,wherein the performance rating is based at least partially on instructorinput.
 7. The method of claim 1, wherein the biofeedback data and theadditional biofeedback data comprise at least one of: collected facialgestures of the operator; captured speech of the operator; and capturedbehavioral and physiological responses of the operator.
 8. A system,comprising: a simulator configured to simulate operation of a craft orvehicle, the simulator comprising: one or more controls and displays,one or more biofeedback sensors, and a crew member simulator configuredto simulate a passenger or crew member of a craft or vehicle; and one ormore processors coupled to the simulator and configured to executeinstructions, stored in one or more memory devices, to perform a methodcomprising: generating, with the crew member simulator, sensorydepictions of a life-sized virtual crew member in the simulator, whereinthe virtual crew member simulates operational actions and responses ofthe passenger or crew member of the craft or vehicle; collecting, fromthe one or more controls and displays and the one or more biofeedbacksensors, data representing a state of the craft or vehicle beingsimulated and one or more responses of an operator using the simulator,wherein the one or more responses comprise biofeedback data associatedwith the operator; generating, in response to at least one of the datarepresenting the state of the craft or vehicle and the one or moreresponses, one or more simulated operational actions or simulatedresponses for the virtual crew member with the crew member simulator;and storing the one or more responses and the one or more additionalresponses as a log of the operator's operation of the simulator.
 9. Thesystem of claim 8, the method further comprising: updating the one ormore simulated operational actions or simulated responses of the virtualcrew member based at least partially on the log of the operator'soperation of the simulator.
 10. The system of claim 8, wherein thesimulator further comprises an instructor station and wherein the methodfurther comprises: receiving, via the instructor station, instructorinput, wherein the one or more simulated operational actions orsimulated responses for the virtual crew member are based at leastpartially on the instructor input.
 11. The system of claim 8, the methodfurther comprising: generating, to the operator, a training summarybased at least partially on the log of the operator's operation of thesimulator.
 12. The system of claim 11, the method further comprising:comparing the one or more responses to an operational standard of thesimulator; determining, based at least partially on the comparison, aperformance rating of the operator's operation of the simulator; andproviding the performance rating as a portion of the training summary.13. The system of claim 8, wherein the crew member simulator comprisesat least one of: an animatronic robot; a virtual reality display; and aholographic display.
 14. The system of claim 8, wherein the biofeedbackdata and the additional biofeedback data comprise at least one of:captured facial gestures of the operator; captured speech of theoperator; and captured behavioral and physiological responses of theoperator.
 15. A non-transitory computer readable medium storinginstructions for causing one or more processors to perform a method, themethod comprising: generating sensory depictions of a life-sized virtualcrew member in an aircraft simulator, wherein the virtual crew membersimulates operational actions and responses of a crew member of anaircraft; collecting data representing a state of the aircraft beingsimulated and one or more responses of an operator using the aircraftsimulator, wherein the one or more responses comprise biofeedback dataassociated with the operator; generating, in response to at least one ofthe data representing the state of the aircraft and the one or moreresponses, one or more simulated operational actions or simulatedresponses for the virtual crew member; and storing the one or moreresponses and the one or more additional responses as a log of theoperator's operation of the aircraft simulator.
 16. The non-transitorycomputer readable medium of claim 15, the method further comprising:updating the one or more simulated operational actions or simulatedresponses of the virtual crew member based at least partially on the logof the operator's operation of the aircraft simulator.
 17. Thenon-transitory computer readable medium of claim 15, the method furthercomprising: receiving instructor input, wherein the one or moresimulated operational actions or simulated responses for the virtualcrew member are based at least partially on the input from theinstructor.
 18. The non-transitory computer readable medium of claim 15,the method further comprising: providing, to the operator, a trainingsummary based at least partially on the log of the operator's operationof the aircraft simulator.
 19. The non-transitory computer readablemedium of claim 18, the method further comprising: comparing the one ormore responses to an operational standard of the aircraft simulator;determining, based at least partially on the comparison, a performancerating of the operator's of the aircraft simulator; and providing theperformance rating as a portion of the training summary.
 20. Thenon-transitory computer readable medium of claim 15, wherein thebiofeedback data and the additional biofeedback data comprise at leastone of: captured facial gestures of the operator; captured speech of theoperator; and captured behavioral and physiological responses of theoperator.